perm filename LOSS[COM,LSP]1 blob sn#827964 filedate 1986-11-11 generic text, type C, neo UTF8
COMMENT ⊗   VALID 00002 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	@center(@b[Issues of Separation in Function Cells and Value Cells])
C00059 ENDMK
C⊗;
@center(@b[Issues of Separation in Function Cells and Value Cells])

@center(@b[by RPG and KMP, etc.])

@b(0. Notation and Terminology) 

This section defines some terminology. Many readers can skip this
section.

In this white paper we will use the following terminology.  An
`identifier' is the name of a symbol or a variable; this is not an exclusive
partition.  An identifier is essentially a print name. A `symbol' is a
Lisp data structure with a value cell, a function cell, and so on. A
`variable' is an identifier that names a location.  A `binding' is a
pairing of an identifier with a location in which location a Lisp object
may be placed. A `lexical variable' is a binding in which the identifier
is not taken to refer to a symbol.  A symbol with a value in its value
cell is taken to be a binding in that the name of the symbol is paired with
the value cell of the symbol. When symbols are used as LET-variables
and as the formal parameters of functions, they are called `special variables.'

An `environment' is the set of all bindings in existence at some given time.
We will call a subset of an environment a `sub-environment.'

A `function' is anything that may correctly be given to the FUNCALL or
APPLY function and is to be executed as code when arguments are supplied.

We will use the terminology, 

``(SETF (SYMBOL-FUNCTION <symbol>) <function>)

associates the function <function> with the symbol <symbol>'' without
regard for such potentially erroneous expressions like this

(SETF (SYMBOL-FUNCTION 'FOO) 'BAZ)

where it is assumed that BAZ has no function definition associated with
it.

A `namespace' is a sub-environment in which the nature of the objects to
which the location part of a binding may point are restricted to some
subset of all possible objects (not necessarily first-class Lisp objects).
In this white paper there are two namespaces of concern, which we will
term the `function' namespace and the `value' namespace.  The function
namespace is a sub-environment whose location parts are restricted to
point to functions, except under error conditions.  The binding of an
identifier referring to a symbol along with a function cell is in the
function namespace.  The binding of an identifier referring to a symbol
along with a value cell is in the value namespace.  Lexical variables,
introduced with LET, LAMBDA, MULTIPLE-VALUE-BIND, and formal parameters
are in the value namespace.  Functional lexical variables introduced by
FLET, LABELS, and MACROLET are in the function namespace.

Other namespaces include tag-names and block-exit-names, but the
objects in the location parts of their bindings are not first-class
Lisp objects.

Lisp evaluations rules tell, given an expression and an environment, how
to produce a value or values and a new environment. In order to do this,
the meanings of identifiers in a program text need to be determined, and
this requires determining in which namespace to interpret the identifiers.
A symbol can have something in its value cell and in its function cell,
and to which of these the name of the symbol refers in an expression
depends on the namespace chosen.

@b(1. Introduction)

In 1981 the emerging Common Lisp community turned to Scheme for some of
its motivation and inspiration. Adopting lexical scoping proved one of the
most important decisions the Common Lisp group every made.  One aspect of
Scheme left behind was the unification of the function and value
namespaces.

The Common Lisp community is now faced with the fact of the existence of
three Lisp communities:  Common Lisp, Scheme, and EuLisp. The latter two
communities have adopted a unified function and value namespace; Common
Lisp has not. The question is whether it is feasible at this point for the
Common Lisp community to change its stand on this point - to merge the two
namespaces - and to merge the Lisp communities.

Before attempting to make any decision on the subject, the Common Lisp
community felt it important to clearly document both sides of the issue
in an unbiased way. This document is a description of the most important
technical and political issues that are involved with this decision.

We hope that the document is neutral in its presentation of these issues.

@b(2. Background)

Historically, most Lisp dialects have adopted a two-namespace approach to
the naming problem. Largely this is because most dialects followed Lisp 1.5
unless there was some interesting reason not to follow it.

Lisp 1.5 broke symbols into values and functions; values were stored on an
a-list, and functions on the property lists of symbols.  Compiled and
interpreted code worked differently from each other.  In the interpreter,
the a-list was where all bindings were kept.  When an identifier was
encountered (an `atomic symbol' in Lisp 1.5 terminology), it was taken to
be a variable to be evaluated for its value. First the APVAL part of the
symbol was interrogated - an APVAL was A Permanent, system-defined VALue
stored in a specific place in the symbol. Second, the a-list was searched.
Finally, if neither of these other two possibilities worked, an error was
signalled.

When a combination was encountered, the function position was evaluated
differently from the other positions. First, the symbol was interrogated
to see whether there was a function definition associated with the symbol,
then the a-list was searched.

Here we can see two namespaces at work, though non-symbol variables (Lisp
1.5 did not have lexical variables in interpreted code) were treated
somewhat uniformly: when there were no function definitions associated with
a symbol, there was one namespace, explicitly represented by the a-list.

Compiled code worked a little differently, and from its internals
we can see where the two namespaces came about in descendants from
Lisp 1.5, at least conceptually.

The Lisp 1.5 compiler supported `common' and `special' variables. A common
variable enabled compiled and interpreted code to communicate with each
other. A common variable was bound on an explicit a-list, and to evaluate
such a variable a call to EVAL was emitted to determine the value.  A
special variable was the compiler's modelling of free variables and
closely matched what is called today `shallow binding.'  `Ordinary'
variables are compiled into what we have termed `lexical variables.'

Thus, we see all of the components of the two-namespace world in Lisp
1.5, along with some of the components of the one-namespace world.

Lisp for the PDP-6 at MIT adopted the style of the Lisp 1.5 special
variables for dynamic binding in both compiled and interpreted code,
eliminating common variables. Compilers were still written to try to
interpret special variables as lexical variables in as many places as
possible. The value of a symbol was stored in the special value cell of
the symbol and the function remained on the property list as it did in
Lisp 1.5.

@b(3. Technical Reasons for the Merger)

@b(3.1 Notational Simplicity)

Many people believe that having the function position be evaluated
differently from the argument positions in Lisp is very inelegant.
The reason that different evaluation rules are needed is because
there are different namespaces or environments for functional bindings
and for value bindings. Therefore there is an evaluation rule for
each environment.  

This feature has the indirect effect of making the notation more
complicated.  There are two things that we need to do:  1.  Fetch the
value of an identifier in the value environment and call it, 2.  Fetch the
value of an identifier in the function environment pass it around as a
value.  The notation for accomplishing 1 is:

(FUNCALL <identifier> . <arguments>)

For example, one might write:
@Begin(Example)
(DEFUN MAPC-1 (F L) (DOLIST (X L) (FUNCALL F X)))
@End(Example)
In a single namespace Lisp, one might write:
@Begin(Example)
(DEFUN MAPC-1 (F L) (DOLIST (X L) (F X)))
@End(Example)

The notation for accomplishing 2 is:

(FUNCTION <identifier>)

This is often abbreviated:

#'<identifier>

For example, one might write:
@Begin(Example)
(MAPC (FUNCTION PRINT) '(A B C D))
@End(Example)
or
@Begin(Example)
(MAPC #'PRINT '(A B C D))
@End(Example)
If there were a single namespace, one might write:
@Begin(Example)
(MAPC PRINT '(A B C D))
@End(Example)

With these two examples we have shown what the notational differences
would be in a two-namespace Lisp versus a one-namespace Lisp.  The
differences are more striking in a larger, more complex example. In Common
Lisp one writes:
@BEGIN(Example)
(DEFUN Y (F)
 (LET ((G #'(LAMBDA (H)
             #'(LAMBDA (Z)
                (FUNCALL (FUNCALL F (FUNCALL H H)) Z)))))
  #'(LAMBDA (X)
     (FUNCALL (FUNCALL F (FUNCALL G G)) X))))
@END(Example)
while in a single namespace Common Lisp, one would write:
@BEGIN(Example)
(DEFUN Y (F)
 (LET ((G (LAMBDA (H)
           (LAMBDA (Z) 
            ((F (H H)) Z)))))
  (LAMBDA (X) ((F (G G)) X))))
@END(Example)
which anyone can see is much simpler.

Probably few people would deny that the syntax resulting from the this
change would make the language appear better visually. If the issue were
being decided purely on the basis of cosmetics, probably few people
would oppose the change.

@b(3.2 Multiple Denotations for a Single Name)

Some people find it less confusing to have a single meaning for a name.
Fewer meanings mean less to remember.

For example, suppose a programmer has defined a function @t(F) as:
@Begin(Example)
(DEFUN F (X) (+ X 1))
@End(Example)
Then suppose he is writing a new function @t(G) and he wants it to
take a functional parameter @t(F) which it is to apply to its other argument.
Suppose he writes:
@Begin(Example)
(DEFUN G (F) (F 3))
@End(Example)
Issues of defined program semantics aside, it's probably obvious that the
programmer who wrote this piece of code meant to call the function named
by the formal parameter @t(F) on the argument @t(3). 
In Common Lisp, this function will ignore its argument named @t(F) and
simply invoke the globally defined function named @t(F) on 3.
Notice that this is precisely what Lisp 1.5 would have done as well.

Unfortunately, not all situations are as clear cut as this. For example, 
in the following example:
@Begin(Example)
(DEFUN PRINT-SQRTS (LIST)
  (DOLIST (ELEMENT LIST)
    (PRINT (LIST ELEMENT (SQRT ELEMENT)))))
@End(Example)
In this example, there are three uses of the name @t(LIST). The first is in
the formal parameter list. The second is in initialization of the @t(DOLIST) 
variable. The third is in the @t(PRINT) expression. This program, which is
valid in current Common Lisp, would not be valid in a Common Lisp where 
functional and argument positions were evaluated in the same way. A common
way to write this code would be:
@Begin(Example)
(DEFUN PRINT-SQRTS (LST)
  (DOLIST (ELEMENT LST)
    (PRINT (LIST ELEMENT (SQRT ELEMENT)))))
@End(Example)

As should be clear from these examples, the advantage of treating the
function and argument positions the same is that using parameters as
functions is made more convenient syntactically.

The disadvantage is that @i(not) using parameters as functions is made
less convenient syntactically, because parameter names must be more
carefully chosen in order to not shadow the names of globally defined
functions which will be needed in the function body.

Notice that only an additional amount of care is needed because
variable names must be carefully chosen to not shadow the names
of variables bound in outer binding constructs. For instance,

(DEFUN HACK-LIST (LIST)
 (LET ((LST (HACK-LIST LIST)))
  (HACK-SOME-MORE LIST LST)
  (SHUFFLE LIST LST)))

In the function PRINT-SQRTS above, the parameter named LST is better named
LIST, though PRINT-SQRTS is not better named PRINT-SQUARE-ROOTS.

The following is a simple example of some of the important issues 
in variable naming:
@Begin(Example)
(DEFUN ODDITY-1 (LIST) (LIST LIST LIST))
(ODDITY #'CONS)
@End(Example)
Depending on which way the issue is decided, the possible return values from
this function might be:
@Begin(Example)
(#<SUBR CONS> . #<SUBR CONS>)
(#<SUBR CONS> #<SUBR CONS>)
@End(Example)

@b(3.3 Higher Order Functions are Possible)

Functions like the Y function above, which manipulate functions directly, are
more perspicuously written within a single namespace Lisp. The point is that
with two namespaces this style is not encouraged, while in a single namespace
Lisp it is encouraged.

@b(3.4 Abstraction Sharing)

In a single namespace Lisp, it is easier to define an abstract piece of
code that shares abstractions between data and functions. An example of
this is the abstract stream code in Chapter 4 of Abelson and Sussman, in
which it is shown how to write streams based either on functions or on
data structures.  Again, all of this is possible in Common Lisp as it
stands, but it is not an encouraged style. The problem is that it is a
burden to think about which namespace will be used for various variables.

@b(3.5 Compiler Simplicity)

In current Common Lisp compilers, special cases code is used when deciding
which namespace mapping to use when a variable is examined by the
compiler. A single namespace Lisp will result in simpler, smaller, faster
compilers in some cases.

@b(3.6 Referential Clarity)

In a two-namespace Lisp it is the case that without a context it is not
possible to decide whether the functional or the value namespace is the
proper one to use. These two forms result in different interpretations of
x:
@BEGIN(Example)
(x ...)
@END(Example)
@BEGIN(Example)
(... x ...)
@END(Example)
Unfortunately, a basic rule of Lisp style is violated, that rule being
that code is clearest when the least amount of context is necessary to
determine what each expression does.

@b(3.7 Multiprocessing)

The functional style which is encouraged by single namespace Lisps
is conducive to multiprocessing. That is, the functional style of
programming results in programs which are more easily rendered into
a parallel style. The evidence for this is to look at typical Common Lisp
programs and contrast their style and suitability for parallelization with
Scheme programs.

@b(4. Political Reasons for the Merger)

@b(4.1 The Scheme Community)

In the US there are two major Lisp communities - Common Lisp and Scheme.
These two communities overlap in terms of people who are proponents of
one of the other dialects depending on the context. For example, Gabriel
is a strong commercial proponent of Common Lisp, yet his own parallel
Lisp research is performed using Scheme as a workbench and starting point.

If Common Lisp were to abandon the two-namespace situation, it would then be
easier to develop a layered definition in which Scheme is the base layer
and Common Lisp is the `industrial strength' layer. Scheme would then be
the natural subset for use on smaller computers and embedded systems.

One would have to be careful not to constrain the Scheme community to
stick too closely to the particular of Scheme that would be this
fundamental layer. In addition, the Scheme community would need to be
convinced that this layering is desirable. Though the entire Scheme
community was not present at any such discussions, Gerry Sussman and Will
Clinger seemed amenable at the last meeting.

@b(4.2 The European Community)

This community will, for the moment, be discussed as if it coincided with
the EuLisp group. This assumption is not, of course, correct, and later we
will argue that that fact can be used to justify retaining the two-namespace
Common Lisp.

The EuLisp group is endeavoring to define a new standard for Lisp - possibly
to be proposed to the ISO community as a standard for Lisp. Their approach is
to take only lessons from the past rather than designs from other dialects of
Lisp. They wish to propose a 3 level standard in which the lowest level,
level 0, is the minimal Lisp, possibly suitable for proving properties of
Lisp programs. The second level, level 1, is of the size and extent of Scheme
and is intended as the right size for small personal computers.
The highest level, level 2, is about of the size and extent of Common Lisp.
It is intended as the industrial strength Lisp with a number of as-of-now
unspecified environmental features.

Rather than building on existing specifications, this group is starting
afresh, with all the concomitant social pressures of developing a
standard using a slightly too large working group. Key member of this
group are Jerome Chailloux, Julian Padget, John Fitch, Herbert Stoyan,
Giuseppe Attardi, and Jeff Dalton.

The key differences between EuLisp as it now stands and Common Lisp are a
single namespace and simple lambda-lists (no &optionals or keywords) for
EuLisp. At one face-to-face meeting, Chailloux and Padget stated that if
Common Lisp were to collapse the two namespaces, they would be willing to
adopt fancy lambda-lists. These two concessions are possibly enough to lay
the groundwork for a widespread set of compromises, mostly from them, on
the merger of the two communities.

Without this technical merger, a battle is at hand. It appears that the
convenorship for ISO Lisp will go to France, with the US announcing its
intentions of taking a technical leadership position. ANSI appears to want
to assuage the European community by throwing a language to them for
convenorship; Lisp is the unfortunate victim.

If the US wanted the convenorship, it might wrestled away from France. Bob
Mathis comments that it is often easier to bend a standard in one's own
direction when technical rather than political leadership is taken.

@b(4.3 The Lisp Community has a Better Commercial Story)

Currently when Lisp vendors are asked how to run Lisp code on small
computers, the answer is either that it isn't possible or that Scheme
is the recommended Lisp for this purpose. This story sounds as if Common Lisp
were not designed to meet commercial needs, because most of the commerical
world uses small computers. If Scheme could be incorporated as a subset
or as a fundamental lower level, the story would be that one could run
Common Lisp, level 0 (that is, Scheme).

@b(5. Technical Reasons Against the Merger)

@b(5.1 Number of Namespaces)

There are really a larger number of namespaces than the two being
discussed here. The interpretation of a symbol in a Common Lisp will
still depend on the context to disambiguate variables from symbols from
type names and so on.

@b(5.2 Macros and Name Collisions)

Macros in Common Lisp are currently written in such a way that
functional variables are assumed constant. That is, suppose there is
a macro defined like this:

@Begin(Example)
(DEFMACRO MAKE-FOO (THINGS) `(LIST 'FOO ,THINGS))
@END(Example)

Here FOO is quoted, THINGS is taken from the parameter list for
the macro, but LIST is free. The macro writer has either assumed
that the referent of LIST in the environment in which the macro
expansion will appear is correct (this is not very likely), or else
he has assumed that LIST is a constant (this is probably the case).

If the programmer writes

@BEGIN(Example)
(DEFUN FOO (LIST) (MAKE-FOO (CAR LIST)))
@End(Example)

in a single-namespace Common Lisp, it is likely that there will
be a bug in the code.

Here is another example:

@Begin(Example)
(DEFMACRO FIRST (LIST) `(CAR ,LIST))
...
(DEFMACRO TEST-CAR (CAR TEST-LIST)
  (DO ((TESTS TEST-LIST (REST TESTS)))
      ((NULL TESTS))
    (FUNCALL (FIRST TESTS) CAR)))
@End(Example)

In some Scheme implementations it is possible to write the following:

@Begin(Example)
(DEFMACRO FIRST (LIST) `(',CAR ,LIST))
@End(Example)

This is syntactically more tedious and that would have to be weighed.
Nevertheless, this proposal doesn't answer all technical problems.  When
uses of this macro is compiled to a file, the object that is output for
the reference to CAR is probably a representation of the procedure object
(compiled code object) for CAR. Possibly not all Common Lisp implementations
are set up to handle the case of outputting a compiled-code object to a
file and then reading it back in.

Consider this piece of code:

@Begin(Example)
(DEFMACRO FOO (N X) `(',(LAMBDA (X) (+ X N)) ,X))
(FOO 3 Z)
@End(Example)

For each use of this macro there may be a separate copy of the
compiled code for the anonymous function in the macro definition.
Some systems are able to collapse structurally equal compiled-code
objects into only one copy when the occurrences of them are in one
file, but this is a non-trivial means of solving part of the problem
with the approach.

The real problem for Lisp implementors to make this approach workable
is that the compiler ought to open-code the original CAR operation
in the TEST-CAR code rather than to code a function call to CAR.
Thus, the compiler will need to be modified to recognize this idiom.

Another solution to the macro problem is based on work by Kohlbecker in
his recent PhD thesis from Indiana. Macros can be written in a style that
is like the current Common Lisp style, but free function position
variables can be defaultly taken to be the globally defined function of
the same name.

A third part of the solution is to include a new declaration form which
declares function definitions constant, and Common Lisp implementations
could declare all Common Lisp functions constant. The correct behavior of
a Common Lisp with the built-in functions declared constant would be to
signal an error when cases like TEST-CAR above occur.

Though this seems like a potentially insurmountable problem - the Scheme
community has not defined a standard macro facility for Scheme because of
this and other problems - Common Lisp already has this problem, but people
do not notice it very much. Here is an example:

@BEGIN(Example)
(DEFMACRO FOO (X Y) `(CONS 'FOO (CONS ,X (CONS ,Y NIL))))

(DEFUN BAZ (X Y)
 (FLET ((CONS (X Y) (CONS Y X)))
  (FOO X Y)))
@END(Example)

(BAZ 1 2) returns (((NIL . 2) . 1) . FOO), even though it seems that
(FOO 1 2) was intended by the programmer (except I, rpg, expected exactly
what happened).

In a correct Common Lisp implementation one can write:

@BEGIN(Example)
(DEFMACRO FOO (X Y) ;take a deep breath
 `(FUNCALL ',#'CONS 'FOO (FUNCALL ',#'CONS ,X (FUNCALL ',#'CONS ,Y NIL))))

(DEFUN BAZ (X Y)
 (FLET ((CONS (X Y) (CONS Y X)))
  (FOO X Y)))
@END(Example)

And (BAZ 1 2) will evaluate to (FOO 1 2), just, um, as everyone expected.
Note that this only works in some implementations of Common Lisp.

Perhaps it is the case that most programmers do not use FLET and LABELS,
though we know of several major user groups who do.

@b(5.3 Space Efficiency)

If a symbol is used both for its value and as a function, it currently costs no
additional space. Any program which has symbols which are used to denote 
distinct functions and values, however, would have to be changed. In general,
this means that some new symbols would be introduced. In most cases, the number
of new symbols introduced would not be extremely large, but there might be
pathological applications where there were exceptions. In the Lucid Lisp system,
there are 14 of these symbols, and the value cell is being used, in these cases,
as a cache for an object related to the function. 

Using the same name to refer to both a function and a value cell can be more
space efficient since it means adding only one additional cell to an existing
data structure that already has on the order of 5 - 10 cells anyway.

This issue can be put qualitatively.  Let N be the number of symbols in
a system, let S be the space occupied by the average symbol in an
implementation of Common Lisp as now defined, let S' be the space occupied
by the average symbol in an implementation of Common Lisp as modified by
merging the value and function environments, and let X be the number of
symbols that must be added to a system to resolve name conflicts.  Then
the space saved by having separate environments is

((N + X) * S') - (N *S)

For example, if N is 8000, X is 14, S' is 28 bytes, and S is 32 bytes,
then the space `saved' is -31608 bytes.  That is, the separate function
environment would take up about 1% more space than the single environment.

In order for there to be no net change in the amount of storage between
a two-namespace and a one-namespace Lisp, one would need over 1100 symbols
to be added to the system to resolve name conflicts.

@b(5.4 Time Efficiency)

In Common Lisp, a function call to a function associated with a symbol
involves indirecting through the symbol's function cell. A typical
implementation on stock hardware will look at the symbol's function cell,
which points to a piece of code, possibly with an intermediate pointer
through a procedure object, as in S-1 Lisp. An optimization to this is for
a function call to jump directly to the correct code object, perhaps
indirecting through a link table of some sort, but eliminating the
indirection through the symbol's function cell.  In order for DEFUN and
(SETF (SYMBOL-FUNCTION...)...)  to cause existing running code to work,
the operation of changing a symbol's function cell will invalidate the
link table or otherwise cause the correct new link to be made.

To use this same optimization in a single namespace Lisp, SETQ, rather
than (SETF (SYMBOL-FUNCTION...)...) must do the invalidating or
re-linking.  The common case is that there is not a function associated
with a symbol - programmers do not often write code that changes function
definitions in inner loops - and so a flag stating there is no function
definition involved will need to be checked on each SETQ.

Of course, only SETQs of symbols need to be checked, so that there could
be a test and branch added to SETQ, where SETQ might have been a single
instruction before. On some stock hardware, tricks with the addressing
hardware and word alignment can be played to make this fast in the
non-functional value case, but in fact it seems unlikely that this could
cause any more than a 10% degradation in the most pessimistic inner loop,
and overall it is unlikely to cause more than a 1% degradation in a large
system.

@b(5.5 Cultural Compatibility)

We have encouraged the naming convention of *...* around global names.
If there were a single namespace, then this convention would need to
be altered to mean that a symbol had a value that was not a global
function definition.

@b(5.6 Special Variable Warnings)

If a single namespace were adopted, symbols naming functions would be,
technically, special variables, and, hence, compilers would normally warn
about them. There are two solutions to this: one is to declare such
functions as constant, and perhaps DEFUN could do this defaultly; the other
is to adopt the notion of a global lexical environment. More will be said
later about global lexical environments.  

There would also need to be a non-constant declaration defined, so that
function names can be dynamically scoped.

@b(5.7 Compatibility Issues)

Making a transition to a world with unified a function/value cell would
involve a considerable amount of incompatibility. There is the question
of implementor problems as well as user problems.

@b(5.7.1 Changing existing code)

Large bodies of code already exist that make assumptions about the current
semantics. That code would all have to be changed. Users who did not favor
this change would likely resent the amount of work required to make the 
change, which might be non-trivial.

In some cases, mechanical techniques could diagnose which programs needed
to be changed. However, because of the pervasive use of macros and of 
automatic programming techniques, it would not be possible to do such
diagnosis with 100% reliability in any automatic way. Compilers could be
modified to provide the user information about conflicts as they come up
as a sort of machine-gun approach to the problem.

@b(5.7.2 Compatibility packages)

Various compatibility schemes have been proposed which would allow these
problems to be, possibly, eliminated.  The simplist idea is to have a
single Common Lisp with the single namespace but with a compiler switch
that allows two-namespace code to run in this image.  Symbols would have
function cells, but possibly represented as properties on property lists.
All old Common Lisp code of the form:

@BEGIN(Example)
(F ...)
@END(Example)

would be transformed to this:

@BEGIN(Example)
(FUNCALL #'F ...)
@END(Example)

where FUNCTION would look things up in the `function cell.' FUNCALL would
be retained in the compatibility package. A bigger example is more convincing:

@BEGIN(Example)
(LET ((LIST ...))
 (LIST ...))
@END(Example)

becomes

@BEGIN(Example)
(LET ((LIST ...))
 (FUNCALL #'LIST ...))
@END(Example)

Presumably a single namespace Common Lisp would retain LABELS, possibly
renamed LETREC, and LET would be used in place of FLET.  During the
transformation process variables bound by occurrences of FLET and LABELS
in the old code would be renamed to GENSYMmed variables, and the value
namespace versions of FLET and LABELS would be substituted for the
function namespace versions.

Possibly some compilers already perform this transformation internally and
will be simplified after the change. And perhaps an implementor will want
to provide a real function cell for this compatibility in order to run old
code relatively fast. Lisps that normally have link tables will need to
provide separate linking code (possibly the old link code) for the
compatibility package.

@b(5.8 Compatibility with Scheme)

Compatibility with Scheme is not currently a goal of the Common Lisp
design committee. On the other hand, if all other things are equal, there
would be no reason for us to seek to be gratuitously incompatible with
Scheme.

However, all other things are not equal. For example, there is an installed
base of code, which would require changing. 

Even if compatibility with Scheme were achieved on this point, there is
currently no iron-clad guarantee that Scheme will not later change in an
incompatible way that will coincidentally thwart such a good faith move on
the part of Common Lisp. It is an explicit goal of the Scheme designers
[ref R↑3 Report] that Common Lisp compatibility not be uppermost in the
list of concerns.

On the other hand, the Scheme designers believe that Common Lisp has a
number of bad warts - two namespaces being foremost - and that, therefore,
compatibility is not as important as it would be if the foundations of
each dialect were more closely akin. With the willingness of the Common
Lisp community to `see the light,' perhaps the willingness to remain
compatible with the newer, merged dialects, would be greater.

If compatibility with Scheme is desired, there are other changes to Common
Lisp that are necessary or desirable. Foremost are tail recursive
semantics and CALL-WITH-CURRENT-CONTINUATION, in which first-class
continuations are supported. This change would be completely upwards
compatible and is not conceptually hard to implement; but, there are many
details and it is not easy to implement.

@b(5.9 Other Changes to Common Lisp to Accomodate the Merger)

A free variable in Common Lisp is taken to be a dynamic rather than
a lexical reference. This is because there is no global lexical environment
in Common Lisp. If this is the code

@BEGIN(Example)
(DEFUN FOO (X)(+ X Y))
@END(Example)

the reference to Y is special (dynamic). On the surface, in a
single-namespace Lisp, the reference to `+' is free also, and so it is
special (dynamic). One proposed solution is to make the default be lexical
(global), which makes Y refer to the global value for Y, and `+' to the
global definition of `+.'

Thus, there would be a global lexical environment in which symbols
that are used freely would be accessed. Currently there is a global
dynamic environment.

This further change would require users to modify their code, the
change being to, possibly, add some SPECIAL declarations.

@b(6. Political (non-technical) Reasons Against the Merger)

@b(6.1 The Lisp Community is Changing its Tune)

The Lisp vendors have convinced the commercial world that the Lisp
community has decided to get its act together and settle on one dialect of
Lisp - Common Lisp - and that it is time to start writing commercial
products using that dialect. If the Lisp community now changes Common Lisp
in such an incompatible way, then the newly-convinced commercial world
might balk.

Perhaps winning the single namespace battle loses the Lisp war.

On the other hand, if X3 is sanctioning this change as part of not
only a US standard Lisp, but an international standard Lisp, perhaps
the change will be excused.

@b(6.2 Can Vendors Afford the Merger?)

Most Lisp vendors have their hands full improving and honing their
products. Typically these vendors schedule tasks 6 months to a year ahead.
At the very least, if the merger takes n person-months to accomplish for
some vendor, there are n person-months of something else very useful that
does not get done.  If the task of the merger is on the order to of
several person-years for a vendor, that vendor might not survive the
change.  If there are such vendors, then the legal protections of CBEMA
for members will be tested.

Gabriel has estimated that at Lucid about 2 person-months are needed to
make the change initially followed by 4 person-months of shakedown.

Vendors like Symbolics will have a lot more work to do to convert their
own system to a single namespace, though they might use the compatibility
scheme mentioned above.

@b(6.3 Can Users Afford the Merger?)

Many users have a lot of Common Lisp code now. Symbolics users are not
reacting well to the switchover to Common Lisp in Release 7, though it was
known for several years that the change would happen. A similar flag day
would be required for the switchover to the single namespace world, but on
a larger scale. [KMP: how expensive was it to manage the switchover?]

Again, perhaps X3 can help by conditioning users for the switchover, and
perhaps DARPA can pay for tools to help users; for example, a codewalker
could be written that could used to parse files and to report places in
the code that require attention.

@b(6.4 The Japanese Community)

The Japanese community seems to want to stick with Common Lisp pretty much
as it is today, according to reports from Prof. Ida. It appears that there
is a heavy commitment to the current definition in the commercial marketplace
there. The political issue seems to be that we will have a battle at the ISO
level regardless of the decision we make.

@b(6.5 Common Lisp can Win the ISO Fight Anyway)

The European Lisp community comprises a number of large and influential
computer manufacturers, many of whom are being convinced that Common Lisp
is a good commercial standard. The US Common Lisp community, including the
large manufacturers who are consumers of Common Lisp, can help persuade these
European manufacturers that a from-whole-cloth Lisp will not make a good
standard, and that the Europeans will be using a standard that is not used
in the US or Japan.

To accomplish this, though, requires some fancy lobbying and politicizing,
to which the Common Lisp community would need to commit.

@b(7. Summary)

There are compelling technical and non-technical reasons on both sides
of the issue. This white paper is an attempt to outline them so that
an informed decision can be reached.